Proactive creation of personalized products

ABSTRACT

A method for proactively creating an image product includes capturing an image of an object in a first environment by a device, storing a library of personalized products each characterized by a product type, automatically recognizing the object in the image as having a product type associated with the library of personalized products, automatically creating a design for the personalized product of the product type using personalized content, automatically displaying the design of the personalized product of the product type incorporating the selected photo in the first environment on the device, and manufacturing a physical product based on the design of the personalized product.

TECHNICAL FIELD

This application relates to utilization of digital images, and morespecifically, to the design and the creation of products based ondigital images.

BACKGROUND OF THE INVENTION

In recent years, photography has been rapidly transformed from chemicalbased technologies to digital imaging technologies. Images captured bydigital cameras and mobile phones can be stored in computers and cloudstorage. Users can view, edit, and print the digital images. The imageprints can be generated locally using output devices such an inkjetprinter or a dye sublimation printer or remotely by a photo printingservice provider. Examples of other image-based products include photobooks, photo calendars, photo mug, photo T-shirt, and so on. A photobook can include a cover page and a plurality of image pages eachcontaining one or more images. Designing an image-based product such asa photobook can include many iterative steps such as selecting suitableimages, selecting layout, selecting images for each page, selectingbackgrounds, picture frames, overall Style, add text, choose text font,and rearrange the pages, images and text, which can be quite timeconsuming. It is desirable to provide methods to allow users to createimage-based or personalized products in a time efficient manner.

SUMMARY OF THE INVENTION

The disclosed methods and systems can significant increase the awarenessin personalized image products such as photobooks. Personalized productsor image-based products using user images are proactively presented tousers, which allow the users to visualize what such products might looklike before they are manufactured. The memories of the users can thus bebetter preserved in more colorful and tangible ways.

Furthermore, the disclosed systems and methods provide convenient waysfor users to obtain designs of image products such as photo books.Images in a group can be automatically arranged into an image productdesign without being asked by a user. The user is provided with anoption to review, to edit, and to request a physical copy of the imageproduct to be made according to the design. As a result, the time andeffort for a user to create and obtain an image product aresignificantly reduced.

In one general aspect, the present invention relates to a method formethod for proactively creating an image product. The method includesstoring a library of specification terms for image products by acomputer system, receiving a voice message from a user, converting thevoice message into a text form by the computer system, conductinglexical analysis of the text, tokenizing the text into a plurality oftokens by the computer system, matching one or more of the tokens to thespecification terms in the library to determine specification parametersfor an image product by the computer system, automatically identifyingimages based on the specification parameters by the computer system, andautomatically creating, by the computer system, a design for the imageproduct that incorporates at least some of the images identified basedon the specification parameters. Implementations of the system mayinclude one or more of the following the specification terms in thelibrary specify one or more image sources from which the images are tobe obtained, wherein one or more of the tokens are matched to thespecification terms in the library that specify the one or more imagesources,

Implementations of the system may include one or more of the following.The step of automatically identifying can include identifying images atthe one or more image sources. The specification terms in the libraryspecify one or more persons, wherein one or more of the tokens can bematched to the specification terms in the library that specify one ormore persons, wherein the step of automatically identifying can includeidentifying images that include the one or more persons in the images.The specification terms in the library specify an occasion that theimages can be associated with, wherein one or more of the tokens arematched to the specification terms in the library that specify anoccasion, wherein the step of automatically identifying can includeidentifying images that are associated with the occasion. Thespecification terms in the library can specify a time period in whichthe images are taken, wherein one or more of the tokens can be matchedto one or more specification terms in the library that specify a timeperiod in which the images are taken, wherein the step of automaticallyidentifying comprises identifying images that are taken in the timeperiod. The specification terms in the library can specify one or moreimage product types, wherein one or more of the tokens are matched to atleast one specification term in the library that specifies an imageproduct type, wherein the design for the image product can beautomatically created in the image product type. The method can furtherinclude selecting a subset of the identified images, wherein a number ofthe identified images in the subset can be within a predetermined rangecompatible with the image product type. The specification terms in thelibrary can specify a style or a format for an image product, whereinone or more of the tokens are matched to at least one specification termin the library that specifies a style or a format for an image product,wherein the design for the image product is automatically created in thestyle or the format. The method can further include manufacturing aphysical manifestation of the image product based on the design for theimage product. The specification terms in the library can specify arecipient for an image product, wherein one or more of the tokens can bematched to a specification term in the library that specifies arecipient for an image product, wherein the physical manifestation ofthe image product can be sent to the recipient after the step ofmanufacturing. The method can further include ranking the identifiedimages using predetermined criteria; and selecting a subset of theidentified images based on a result of the ranking step, wherein thedesign for the image product incorporates the subset of the identifiedimages. The method can further include after the step of matching,presenting a question to the user by the computer system; receiving ananswer from the user; tokenizing the answer into a plurality of tokensby the computer system; and matching one or more of the tokens to thespecification terms in the library to determine additional specificationparameters for an image product by the computer system. The step ofautomatically creating can include assigning at least a portion of theidentified images into a first page group and a second page group eachcomprising one or more images; placing the first page group of one ormore images in the first page of the image product; and placing thesecond page group of one or more images in the second page of the imageproduct. The step of conducting lexical analysis of the text can includeconducting parsing or syntactic analysis of the text by the computesystem. The step of tokenizing can include parsing the text. The step ofmatching can include matching the one or more of the tokens tomorphological variations of the specification terms in the library todetermine specification parameters for an image product by the computersystem.

In another general aspect, the present invention relates to a method forproactively creating an image product. The method includes storing alibrary of specification terms for image products by a computer system;receiving a command from a user; tokenizing the command into a pluralityof tokens by the computer system; matching one or more of the tokens tothe specification terms in the library to determine specificationparameters for an image product by the computer system; automaticallyidentifying images based on the specification parameters by the computersystem; and automatically creating, by the computer system, a design forthe image product that incorporates at least some of the imagesidentified based on the specification parameters.

Implementations of the system may include one or more of the following.The specification terms in the library can specify one or more imagesources from which the images are to be obtained, wherein one or more ofthe tokens can be matched to the specification terms in the library thatspecify the one or more image sources, wherein the step of automaticallyidentifying can include identifying images at the one or more imagesources. The specification terms in the library can specify one or morepersons, wherein one or more of the tokens are matched to thespecification terms in the library that specify one or more persons,wherein the step of automatically identifying can include identifyingimages that include the one or more persons in the images. Thespecification terms in the library can specify an occasion that theimages are associated with, wherein one or more of the tokens arematched to the specification terms in the library that specify anoccasion, wherein the step of automatically identifying can includeidentifying images that are associated with the occasion. Thespecification terms in the library can specify a time period in whichthe images are taken, wherein one or more of the tokens are matched toone or more specification terms in the library that specify a timeperiod in which the images are taken, wherein the step of automaticallyidentifying can include identifying images that are taken in the timeperiod. The specification terms in the library can specify one or moreimage product types, wherein one or more of the tokens are matched to atleast one specification term in the library that specifies an imageproduct type, wherein the design for the image product is automaticallycreated in the image product type. The method can further includeselecting a subset of the identified images, wherein a number of theidentified images in the subset is within a predetermined rangecompatible with the image product type. The specification terms in thelibrary can specify a style or a format for an image product, whereinone or more of the tokens are matched to at least one specification termin the library that specifies a style or a format for an image product,wherein the design for the image product is automatically created in thestyle or the format. The method can further include manufacturing aphysical manifestation of the image product based on the design for theimage product. The specification terms in the library can specify arecipient for an image product, wherein one or more of the tokens arematched to a specification term in the library that specifies arecipient for an image product, wherein the physical manifestation ofthe image product is sent to the recipient after the step ofmanufacturing. The method can further include ranking the identifiedimages using predetermined criteria; and selecting a subset of theidentified images based on a result of the ranking step, wherein thedesign for the image product can incorporate the subset of theidentified images. The method can further include after the step ofmatching, presenting a question to the user by the computer system;receiving an answer from the user; tokenizing the answer into aplurality of tokens by the computer system; and matching one or more ofthe tokens to the specification terms in the library to determineadditional specification parameters for an image product by the computersystem. The step of automatically creating can include: assigning atleast a portion of the identified images into a first page group and asecond page group each comprising one or more images; placing the firstpage group of one or more images in the first page of the image product;and placing the second page group of one or more images in the secondpage of the image product. The step of receiving a command can includereceiving a text form the user, wherein the text is tokenized by thecomputer system. The step of receiving a command can include receiving avoice message from the user; and recognizing a text in the voicemessage, wherein the text is tokenized by the computer system. The stepof tokenizing can include parsing the command. The step of matching caninclude matching the one or more of the tokens to morphologicalvariations of the specification terms in the library to determinespecification parameters for an image product by the computer system.

In another general aspect, the present invention relates to a method forproactively creating a photobook. The method includes identifying agroup of images by a computer system; automatically creating a designfor an image product by the computer system without receiving a user'srequest for designing the image product if the number of images in thegroup is determined to be within a predetermined range, wherein theimage product incorporates one or more images in the group; presentingthe design of the image product to a user for preview; and receiving anorder from the user for a physical manifestation of the image productbased on the design.

Implementations of the system may include one or more of the following.The group of images can be received in an upload from a computer deviceto the computer system. The group of images can be formed in anelectronic share from a first user to a second user. The group of imagescan be clustered when they are selected for printing. The group ofimages can be clustered when the images are placed in an electronicfolder or an electronic album. The group of images can be tagged by acommon label. The method can further include automatically selecting aformat or a style for the design of the photobook before the step ofautomatically creating a design for a photobook by a computer system.The method can further include determining if the number of images inthe group is determined to be within a predetermined range, wherein thepredetermined range varies depending on if the group of the images isclustered in an upload from a computer device to the computer system, orwhen the images are selected for printing, or when the images are placedin an electronic folder or an electronic album, or when the images areplaced in an electronic folder or an electronic album, or the images aretagged by a common label. The image product comprises a plurality ofpages. The step of automatically creating a design for an image productcan further includes: assigning at least a portion of the plurality ofimages into a first page group and a second page group each comprisingone or more images; placing the first page group of one or more imagesin the first page of the image product; and placing the second pagegroup of one or more images in the second page of the image product. Thestep of automatically creating a design for an image product can furtherincludes: automatically selecting a first page layout from a library ofpage layouts, wherein the first page group of one or more images areplaced in the first page according to the first page layout; andautomatically selecting a second page layout from the library of pagelayouts, wherein the second page group of one or more images are placedin the second page according to the second page layout. The method canfurther include allowing the user to select the format or the style forthe design of the image product after the step of presenting the designof the image product. The method can further include making the physicalmanifestation of the image product in accordance to the design. The stepof presenting the design of the image product to a user for preview caninclude sending an email message to the user to present the design ofthe image product. The image product can include multiple pages thateach is configured to display one or more images in the group of images.The image product can include a single page that is configured todisplay one or more images in the group of images.

In another general aspect, the present invention relates to a method forproactively creating a photobook. The method includes identifying agroup of images by a computer system; automatically selecting a formatand a style for an image product by the computer system if the number ofimages in the group is determined to be within a predetermined range;automatically creating a design for the image product in the selectedformat and the style by the computer system without receiving a user'srequest for designing an image product using the group of images,wherein pages of the image product incorporates one or more images inthe group; presenting the design of the image product to a user forpreview; and receiving an order from the user for a physicalmanifestation of the image product based on the design.

In another general aspect, the present invention relates to a method forproactively creating a photobook. The method includes identifying agroup of images by a computer system; automatically creating a designfor an image product by the computer system without receiving a user'srequest for designing the image product incorporating one or more imagesin the group if the number of images in the group is determined to bewithin a predetermined range; presenting the first design of the imageproduct to a user for preview; detecting a change in the group ofimages; automatically modifying the first design of image product tocreate a second design for the image product in response to the changein the group of images; and presenting the second design of the imageproduct to a user for preview.

Implementations of the system may include one or more of the following.The change in the group of images can include addition to the group ofimages or removal of images from the group of images. The change in thegroup of images can include a change in an image property of the groupof images. The image property can include a tag, a keyword, a title of afolder where the images are placed, or metadata at least one of theimages.

BRIEF DESCRIPTIONS OF THE DRAWINGS

The accompanying drawing, which are incorporated in and form a part ofthis specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention:

FIG. 1 is a block diagram of a system for imaging applicationscompatible with the present invention.

FIG. 2 shows a flow chart for proactively creating a photobook inaccordance to the present invention.

FIG. 3 is an exemplified user interface showing an image album having aplurality of images.

FIG. 4 is an exemplified user interface showing an image album having aplurality of images, and an indication that a photobook has beenproactively created.

FIGS. 5A and 5B illustrate an exemplified user interface for showing aproactively created photobook.

FIG. 6 shows a flow chart for proactively creating a design for aphotobook in accordance to the present invention.

FIG. 7 illustrates sorting and grouping of images by image capture timein accordance to an aspect of the present application.

FIG. 8 illustrates sorting and grouping of images by image color contentin accordance to another aspect of the present application.

FIG. 9 illustrates sorting and grouping of images by image-capturelocation in accordance to another aspect of the present application.

FIG. 10 shows a flow chart for proactively creating an image product inaccordance to some embodiments of the present invention.

FIG. 11 shows a flow chart for proactively creating an image product inaccordance to some embodiments of the present invention.

FIG. 12 shows a flow chart for proactively creating an image product inaccordance to some embodiments of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

In the present invention, the term “photo album” refers to a book thatincludes one or two cover pages and one or more image pages. Each of theimage pages can include one or more images. The image pages can alsoinclude text or image caption. The image layout can also include abackground pattern. In the imaging industry, photo albums can includephoto books, scrapbooks, snap books, and any books form including boundimage pages.

A computer 100 illustrated in FIG. 1 represents an exemplified hardwaresetup for executing software that allows a user to perform tasks such ascommunicating with other computer users, accessing various computerresources, and viewing, creating, or otherwise manipulating electroniccontent, that is, any combination of text, images, movies, music orother sounds, animations, 3D virtual worlds, and links to other objects.The system includes various input/output (I/O) devices (a mouse 103, akeyboard 105, a microphone 106, and a display 107) and a computer 100having a central processor unit (CPU) 121, an I/O unit 117 and a memory109 that stores data and various programs such as an operating system111, and one or more application programs 113. The computer 100 alsotypically includes non-volatile memory 110 (e.g., flash RAM, a hard diskdrive, and/or a floppy disk or other removable storage media) and acommunications card or device 123 (e.g., a modem or network adapter) forexchanging data with a network 127 via a communications link 125 such asa cable modem, DSL service or wireless Internet connection. The computer100 of FIG. 1 also can be connected to various peripheral I/O devices.One of the more popular of such peripheral devices is a digital camera108. The digital camera 108 enables users to take pictures (i.e.,images), which are saved in memory within the digital camera 108 in adigital format. After taking and storing the images, the user canconnect the digital camera 108 to a computer 100 in order to upload thedigital images to the computer's disk drive or other non-volatile memory110. The user can also wirelessly transfer digital images in the digitalcamera to the computer 100. Once the digital images are uploaded to thecomputer 100, the user can erase the digital images from the memory ofthe digital camera 108 so that the user can take and store additionalimages using the digital camera 108. Typically, the digital camera 108is connected to the computer 100 only while the user is uploading imagesto the computer's disk drive or other non-volatile memory 110. Once thedigital images are stored on the computer 100, a user can performvarious operations on them. For example, an image viewer application canbe used to view the images. A photo editing application can also be usedto touch-up or otherwise modify the images. The computer 100 cancommunicate via Internet 150 with a server 160 provided by a serviceprovider, such as Shutterfly, Inc. A user operating the computer 100 canupload or download images between the computer and the server 160. Theuser can also electronically share his or her images facilitated by theservice provider. The server 160 can be in communication with a database165 that is also operated by the service provider. The database 165 canstore account information, and product design, product creation andproduct order information in association with the user.

It should be noted that the computer 100 can exist in otherconfigurations from the example shown in FIG. 1 and described above. Thecomputer can include a mobile device such as a smart phone, a digitalassistant device, and a wireless enabled mobile computer such as alaptop computer or a tablet computer,

Although large numbers of images are frequently captured by digitalcameras and mobile phones, people often are not aware of the customizedimage products that are available from image service providers such asShutterfly Inc. Customizable image products or personalized productinclude photobooks, photo calendars, personalized greeting cards, photostationeries, photo mugs, photo T-shirt, photo or image prints, photoposters, photo banners, photo playing cards, photo aprons, or magnets,mouse pads, phone cases, cases for tablet computers, key-chains, photocollectors, coasters, banners, or other types of photo gifts thatinclude personalized content. Other examples of personalized productsinclude glassware, tiles, woodwork, jewelry trays, paperweights,pillows, and mugs that include customizable graphic designs, text, orimages. These personalized products provide tangible ways for preservingpeople's treasured memories in addition to viewing images on electronicdisplays.

Additionally, significant obstacles exist even for people who are awareof the availability of customizable image products. The design of aphotobook, for example, can take significant amount of time and effort.A user has to select a large number of quality and emotionallysignificant photos, increasing from different sources, such as, a localcomputer or phone, at a data storage operated by the service provider,or at a third party social site (e.g. Facebook™, Instagram™ . . . ). Theuser needs to select a format, and a style for the photobook. The userneeds to design or select layout for each page including a cover page,sort images in different page groups, and place the images ontoindividual pages. The photobook design can take many iterative stepssuch as selecting suitable images, selecting layout, selecting imagesfor each page, add text, which can often take hours to complete.

Once the user approves a design for an image product, the image product(e.g. a photobook) can be made according to the design locally by aprinter 130 connected to the computer 100, or remotely by the imagingservice provider at a printing facility 170.

It should be understood that the presently disclosed systems and methodsare suitable for proactively creating different image products otherthan photobooks. The image products can include multiple pages (i.e.faces or views) that each can display one or more images, such as, photobooks, photo calendars, photo scrapbooks, photo snapbooks, photocalendars, photo albums, a multi-face photo card. The image products canalso include a single page or view for displaying one or more images,which can include, for example, photo greeting cards, photo calendars,holiday cards, photo mugs, photo T-shirts, photo aprons, single photopages, photo collage pages, a photo stationery, photo banners, photomugs, photo mouse pads, photo key-chains, photo collectors, and photocoasters etc.

In some embodiments, referring to FIG. 2, an image product can be madeavailable to a user in the following process. It should be noted thatthe process can include substeps within each step and other steps notshown in FIG. 2. Additionally, certain steps in the process may beoptional.

A group of images is identified by a computer system such as thecomputer 100 or the server 170 (step 210). The images can be received ina group in an upload from a computer device (e.g. the computer 100,FIG. 1) to the computer system (e.g. the server 170, FIG. 1), or from adigital camera to a computer (e.g. the computer 100, FIG. 1). The imagescan form a group when the images are shared in an electronic messagefrom a first user (e.g. operating the computer 100, FIG. 1) to a seconduser via a computer server (e.g. the server 170, FIG. 1). The images canbe identified when a group of images is selected for printing. Theimages can also be identified when the images are placed in anelectronic folder or an electronic album for an occasion or event, orsimply for collecting pictures of a theme or interest (which can becalled “a picture picker”). As shown in FIG. 3, a user interface 300(e.g. a web browser) displays an image album 310 that can have an albumname 315 such as “Graduation Celebration” and a plurality of images 320.The image album 310 can include several pages of images that a user cannavigate to and view by clicking arrow buttons 340, 350. Furthermore,images may be tagged by a user or the service provider by labeled suchas “Hawaiian Vacation”, “Soccer Team”, “Birthday Party”, “Favorite”,etc. A group of images can be identified when they are tagged by acommon label such as “Hawaiian Vacation”.

The computer system then determines if the number of images is within apredetermined range (step 220). For example, the threshold for thenumber of images in the group can be 20 (i.e. the predetermined range is20 or higher). If the images in the group are within the predeterminedrange, the computer system automatically selects a style and a formatfor an image product (step 230). For example, the styles of the imageproducts can include, romance, seasonal, everyday, vacation, travel,wedding, baby, kids, birthday, school memories, sports, Mother's day,holiday, etc., each of which can include more detailed styles. In theexample of photobooks, the style of a photobook can specify the theme ofthe photobook such as colors, the background, patterns and graphics laidon the page, picture borders, the text font, positions of the text, andthe images on the covers of the photobook. The format of a photobook caninclude the size e.g. 5″×7″, 8″×8″, 12″×12″ etc.), the paper stockmaterials for the pages and the cover, the type of binding, and thefinish of the photobook.

In some embodiments, the predetermined range can depend on how the groupof images is selected, whether they are selected for printing, or forelectronic share, or placed in a single electronic folder or album, orreceived in an image upload, or tagged by a same label. For example, thethreshold number can be 15 for the images that are selected forprinting, and 20 for the images that are selected for electronicsharing. The different threshold image numbers for different imageaggregation sources is based on the observation that users conductingdifferent image operations often have different degrees of interests inseeing the design and making a physical image product such as aphotobook. The above described flexibility in image threshold numberscan provide service best suitable to different segments of users.

In some embodiments, the predetermined range can depend on the type ofimage products. For example, a calendar book may require 12 differentimages for 12 months plus an extra image for the calendar cover. Sincethe calendar cover may use a redundant image from the months, thereneeds to be at least 12 images for a calendar book. Therefore theminimum number of images for a 12-month calendar book is 12. Similarly,the minimum number of images for an 18-month calendar book is 18. On theother hand, the design of a photo mug can be proactively prepared whenthere is at least one image in the group. A single-page photo collagemay require a minimum of M×N images if the collage includes at least Mrows and N columns, wherein M and N are positive integers.

The style and the format of the image product can have defaultselections. For example, a photobooks can have “everyday” and hardcoverfor a book style, and 8″ by 8″ for the photobook format. In someembodiments, the style and the format of the photobook can be selectedbased on knowledge of the images in the group. Specifically, an imageproperty can be extracted from the group of identified images. Forexample, if the images are identified by a common tag label “HawaiiVacation”, a photobook style for vacation, or specifically for Hawaiianvacation, may be automatically selected. In another example, if theimages identified are in an electronic album called “Molly's BirthdayParty”, a birthday photobook style can be automatically selected. Inanother example, the album name 315 (FIG. 3) in which the images (320,FIG. 3) are identified can be used to automatically select a photobookstyle (such as a “Graduation” photobook style).

The computer system automatically produces a design incorporating theidentified images using the automatically selected style and format forthe image product (step 240). It should be noted that the design of theimage product using the group of identified images, and the associatedselections of style and formats, are proactively and automaticallyimplemented by the computer system without receiving a request from auser. The approach described in the present systems and methods isdifferent from conventional processes for crating image products, whichrequire users to initiate projects to design the image products. Asdescribed above, the conventional methods require awareness andfamiliarity of the image products by the users. The projects of creatingimage products such as photobooks can be very time consuming. A lot oftime must be committed by a user without her knowing how an imageproduct could look like at the end.

The proactively and automatically created design of the image productsuch as a photobook can be presented by the computer system to a userfor preview (step 250). For example, as shown in FIG. 4, the userinterface 300 can display a button 400 “See Your Photobook” to indicatethat a photobook based on the images 320 in the image album 310 has beenproactively created, and ready for the users view. A photobook design500 is displayed in the user interface 300 after the user clicks thebutton 400, as shown in FIGS. 5A and 5B. The photobook can include afront cover 510, a back cover 520, and a plurality of pages 530, 540.The book title can be automatically selected from the name of the imagealbum 310 (FIG. 3). The front cover 510 can include a book title 515 andan image 518 selected from the images 320 (FIG. 3). For example, theimage 518 can be the first image in the album 310 or the image used (fora thumbnail image) to represent the image album 310. The page 530 caninclude an image 531. The page 540 can include images 541, 542, and text543. The presentation of the photobook design 500 can include realisticillumination and texture to imitate effects of the materials in theselected book format.

The user can click arrow buttons 550, 560 to view different pages of thephotobook. The cover and different pages of the photobook design 500 canalso be played automatically like a slide show once the user clicks thebutton 400 (FIG. 4).

Optionally, the user is allowed to edit the design of the image productsuch as the photobook design 500 (step 260). The user can change,switch, or remove the images 518, 531, 541, and 542 on the book cover510 and pages 530, 540. The user can also change text information suchas the book cover 515 and the text 543. It should be noted that thephotobook design 500 represents a complete design of a photobook that isready to be ordered. The editing is an optional step depending on theuser's time availability.

It should be understood that the proactively created designs of theimage product can be presented in many forms. As describe above, imagesidentified to create a photobook design typically belong to a group,such as a group of shared images, posted images, uploaded images, orimages selected for printing, etc. The proactively created photobookdesign is presented to the user at relevant places on the web pages,such as on a web page for confirming the share of the group of images,at a share web when the group of shared images are viewed, or at a pageafter a group of images have been ordered for printing. The imageservice provider can also proactively send an email to inform someoneabout the creation of the photobook design. For example, a user thatjust uploaded a group of images to the server managed by the imageservice provider may receive an email message from the service providerabout the creation of the photobook design.

After the user has reviewed and optionally edited the photobook design500, the user can save the design of the image product (e.g. thephotobook design 500) and order a physical manifestation of the imageproduct (e.g. the photobook) based on the to edit the design of theimage product (e.g. the photobook design 500) from the image serviceprovider (step 270).

The image service provider can receive the order via the server 160 andmanufacture a physical manifestation of the image product at theprinting facility 170 according to the design of the image product (e.g.the photobook design 500) (step 280).

Details about the proactive design of a photobook (in steps 230-240,FIG. 2) are described below. Referring to FIG. 6, a group of imageshaving numbers within a predetermined range are first obtained asdescribed above by a computer system (step 600). A style and a formatcan be automatically selected for a photobook design (step 610). Theimages are sorted by one or more criteria to produce a desirablesequence that the images will be presented in a photo book (step 620).The criteria can include a primary criterion. For example, the imagescan be sorted by their associated image capture times. The image capturetimes can be extracted from the header information in the image files.For example, a JPEG image may include an EXIF header that includes thecapture time of the image. It is stored in the image file in associationwith the image data by the digital camera. Referring to FIG. 7, acumulative image count can be plotted against image capture time. In oneimplementation, the list of images can be sequenced chronically based oncapture time of the images. Users often prefer to have images to appearin book pages in a chronological order. The images can be then separatedinto page groups using one or more secondary criteria. For example,images can be grouped according to capture time. Images captured in ashort time interval are more likely suitable to be placed on the samepage, facing pages, or adjacent pages in a photo book. In someembodiments, images that are taken in a short interval (e.g. a day) canbe assigned to a same group (e.g. Group A, FIG. 7). The images in GroupA, Group B, or Group C (FIG. 7) are captured in a short time interval.The images in each of these groups can be respectively grouped ascandidates to appear on a same page, facing pages, or adjacent pages.

The sorted images may be first placed into a group including a largenumber of images that do not fit on a page. For example, ten images inGroup B (FIG. 7) are taken in a short time interval such a day. Theimages in Group B can be further separated into page groups using othersecondary criteria such as color content and image capture locations.For instance, the most popular color of each of the ten images can beplotted in a color space (such as (Y, u, v), (L, a, b) etc.), as shownin FIG. 8. The color space is first separated into finite number ofregions. The most popular color is defined as the color region whereinthe most pixel values fall. The ten images can be further sub-groupedaccording to their adjacencies in the color plane (u, v): images 1-3 maybe categorized into a first group; images 4-6 may be categorized into asecond group; images 7-10 may be categorized into a third group. Theimages 1-10 in Group A might have been taken within a same day. Thephotographer might have traveled to several different surroundings, suchas a beach, a restaurant indoor, or a swimming pool at the hotel, eachof which may have different color distributions. It should be understoodthat the display of color content distributions of images are notlimited to a specific color coordinates or a two-dimensional color planeshown in FIG. 8. The color distributions can be plotted in threedimensional color spaces (Y, u, v), (L, a, b), or two dimensional colorplanes such as (Y, u), (Y, v), (a, b), (L, a) etc. In some embodiments,the images with close distance in color space are intentionallyseparated across different pages in the photobook such as the images ona page will not look alike.

In some embodiments, the images can be sorted or grouped based ongeographic locations. For example, assuming Group C (FIG. 7) includesfive images that are taken, for example, in a same day. Geo locationinformation about each of the five images can be extracted from theheader information of the five images, and plotted as shown in FIG. 9.Images 1-3 are taken in close proximity while images 4-5 are taken atsimilar locations. Images 1-3 can be sub-grouped together and assignedto Page 4 of the photo book. Images 4-5 can be sub-grouped together andassigned to Page 5 of the photo book. In some embodiments, the Images1-3 and the Images 4-5 can be assigned to be on facing pages (e.g. suchas pages 4 and 5) such that the images from the same events can beviewed on facing pages in one open-book view. In some embodiments, theimages with close geographic proximity are intentionally separatedacross different pages in the photobook such as the images on a pagewill not look alike. The placements of two page groups of images onfacing pages can be based on one or more secondary criteria, such asimage capture time, image color content, and image capture location.

A page layout is automatically selected from the library of page layoutsin accordance with the properties of the images on the page as well asthe text information associated with the image receiving areas in thepage layout (step 630). The format of the image receiving field in thepage layout matches the format of the selected image. For example,images in Group A (FIG. 7) may include two landscape images (L) and twoportrait images (P). A notation for the image formats for Group A can be“LLPP”. The page layout selected from the page-layout library shouldhave four image receiving areas with matching image formats “LLPP”.

The page layout also includes an image caption area positioned adjacentto the image receiving area for receiving the associated textinformation. For example, one portrait image and one landscape image maybe selected for the next page. The portrait image has associated textinformation entered by the user and stored in conjunction with theportrait image. The landscape image is not associated with any textinformation. In selecting a page layout, the disclosed system firstselect page layouts containing two image receiving areas, and thenfurther selects page layouts that have include one portrait and onelandscape image receiving areas. Additionally, the disclosed systemfurther selects page layouts that include an image caption area inassociation with the portrait image receiving area. After the pagelayout is selected, the selected image(s) and associated text areautomatically placed in the selected page layout (step 640). Theplacement of the selected image(s) and associated text can be done in asingle step because they are pre-stored in an integrated data structureas described previously. The same steps can be repeated for fill theimage fields and text fields in each page of a photo book. The user canselect an image from the plurality of images to be used as part of thebook cover. Details about arranging images in page layouts are disclosedin the above referenced commonly assigned U.S. patent application Ser.No. 12/406,873, titled “Smart photobook creation”, the content of whichis incorporated herein by reference.

In some embodiments, the design of an image product can be dynamicallychanged as the identified images change in the group. For example, thedesign of the image product can dynamically change as a user adds orremoves images in and out of an image album (310, FIG. 3). The number ofpages, the number of images per page, and the layout of images on thepages in a photobook, for example, can automatically adapt to thechanges to the images identified in the group. In general, the processof updating the design of an image product can include the followingsteps: similar to the descriptions above in relation to steps 210-250(FIG. 2), referring to FIG. 10, a group of images is first identified(step 1010). If the number of images in the identified group of imagesis determined to be in a predetermined range (step 1020), a style and aformat are automatically selected for the design of an image product(step 1030). A first design of an image product is automatically createdwithout receiving a request from a user (step 1040). The first design ofthe image product is made available for the preview by a user (step1050). When a change is detected in the group of identified images (step1060), a second design of the image product is automatically createdwithout receiving a request from a user in response to the changedetected in the group of images (step 1070). The change in the group ofimages can include addition to the group of images or removal of imagesfrom the group of images. The change in the group of images can alsoinclude a change in an image property of the group of images. The imageproperty can include a tag, a keyword, a title of a folder where theimages are placed, or metadata at least one of the images.

The second design of the image product is subsequently presented foruser's preview (step 1080). The change in the group of identified imagescan include addition, removal, and a change in the sequence, addition ofremarks and keywords, addition in change in tagging in a group ofimages. The uploading of images into an image folder/album can take morethan one uploading events. The design of the image product can becontinually updated as new images are received by the imagefolder/album.

In some embodiments, referring to FIG. 11, a library of specificationterms for image products is stored in a computer system (step 1110). Thecomputer system can include a network based computer system, a userdevice, a combination thereof, or in other types of configuration. Thespecification terms cover all necessary information for creating animage product. For example, the specification terms include the names ofimage products or image services, or the types of image products andservices, such as photobook, greeting card, calendar, photo mugs, photoT-shirts, photo aprons, single photo pages, photo collage pages, a photostationery, photo banners, photo mugs, photo mouse pads, photokey-chains, photo collectors, photo coasters, or a “photo story”, phonecase, and various photo gifts and photo novelty items, etc. The librarycan store variations of the specification terms. For example, aphotobook can also be referred to as a scrapbook or a book. A greetingcard can have different variations such as a photo greeting card, aholiday card, a Christmas card, a birthday card, a Valentine card, aninvitation card, a thank you card, an announcement card, a get-wellcard, etc. Other examples of image products and services are availableat websites operated by Shutterfly™, Tinyprints™, MyPublisher™,ThisLife™, etc.

The specification terms can also include contact names or equivalentassociated with a user or in a user's address book, which can be storedin the user's computer device or at a central location operated by aservice provider. The contact names can include family members, friends,and acquaintances. For example, the person grandma is defined in contextof the user. The contact names can also include faces recognized in theuser's images stored in the user device(s), at an image service providersuch as Shutterfly™, Inc., or at a third party service provider such asFacebook™, Instagram™, etc. In relation to image products, the contactnames can be the subjects in the photos to be incorporated in an imageproduct, or the recipients for whom the image product is made.

The specification terms can include time period within which the imagesare to be selected from. A photobook for the year 2013 usuallyincorporates images taken in 2013. Unless specified, a holiday cardrefers to the current holiday season or the holiday season coming up.The specification terms can include occasions for which an image productis to be created, for example, Christmas, wedding, birthday, soccergame, anniversary, vacation, trip to the Caribbean's.

The specification terms can include product styles such as modern,romantic, tropical, or styles developed by designer. The specificationterms can include color preferences such as warm, cool, blue, red,green, etc.

The specification terms can define the sources, from which the imageswill be obtained. For example, the image sources can include userdevices such as a phone, a user desktop computer, or a back drive inuser's home, image service provider such as Shutterfly™, third partyservices such as Facebook™, Instagram™, Pinterest™, and cloud storageservices such as Amazon™, Google™, Dropbox™, Carbonite™, etc.

The specification terms can include one or more action words in a usercommand for creating an image product. Examples of action words include“make”, “create”, “design”, “form”, “develop”, “produce”, “order”,“buy”, etc. These action words are often used at the beginning of a usercommand, and can be used to clearly identify a user command for creatingan image product.

A command is received from a user (step 1115) by the computer system.The command includes a text formed by a string of characters, which canbe input by a user, or recognized from a voice message of a user. Theuser command is typically in a natural language comprising words in astring of characters. Examples for a user command include “Create aphotobook for 2013”, or “Make a 2014 photo calendar for grandma”, or“Make a photobook for Peter's soccer team”.

A lexical analysis is conducted on the command (step 1120) by thecomputer system. The lexical analysis can include parsing or syntacticanalysis of the text in the user command. The text string in the usercommand is tokenized into a plurality of tokens (step 1125).

The tokens obtained from the user command are then matched by thecomputer system to the specification terms in the library to determinespecification parameters for an image product (step 1130). The step ofmatching can include first converting the tokens to the standard formsto be consistent to the specification terms stored in the library, whichcan include removing the article the” or “a” in the front of a token,simplifying plural to a singular form, identifying synonyms ormorphological variations of a standard term and converting them tostandard forms. The step of matching then involves matching the tokensin their standard forms to the specification terms in the library.

The matched specification terms can determine specification parametersfor an image product. For example, tokens “photobook” and “2013” in theuser command “Create a photobook for 2013” define a photobook thatinclude photos taken in the year 2013. These action words such as “make”or “create” can be used to identify a user command for creating an imageproduct.

Optionally, the user can be asked by question(s) by the computer systemto define additional specification parameters for the image product(step 1135). For example, in the user command “Create a photobook for2013”, the additional question can include “For whom the photobook willbe made?” or “Is the photobook for the family?” (i.e. “will only photosof family members be included?”). Some messages like “Create aphotobook” or “Make a birthday card” do not provide sufficientinformation to specify an image product. The computer system can promptquestion to the user to clarify information such as the time period, thesubject in the photos of the photobook, whose birthday the card is for,from whom, etc. User's answers to these additional questions areanalyzed similar to the original user command in steps 1115-1130 above.

An image product type is automatically determined by the computer systembased on the matched specification parameters that correspond to aproduct type such as “photobook” or “birthday card” in the library (step1140).

Additionally, a style or a format for the image product can beautomatically selected based on the specification parameters (step1145). For example, if the user command or user answers to questionsinclude “8 by 8 inch photobook”, “wedding book”, “folded greeting card”,or “two piece iPhone 5s case”, an 8″×8″ photobook, a premium weddingstyle photobook, a 5″×7″ folded greeting card, and a two-piece phonecase for iPhone 5s can be respectively selected automatically.

Images are automatically identified and selected based on thespecification parameters (step 1150), which can include severalsub-steps.

First, images may be evaluated in a subset of or all available imagesources. If a user command such as “Create a photobook for 2013” doesnot specify the sources of the images the image product is toincorporate, the images satisfying the criteria (e.g. taken in 2013)from all available image sources can be evaluated. Alternatively, theuser command or answer may include image source specifications such as“Create a photobook for 2013 using pictures in my Shutterfly account”,or “Create a photobook for 2013 with photos from Shutterfly, Facebook,and Instagram”. Images will be retrieved and evaluated in the specifiedimage sources. As described above, image sources can include the imagesources can include user devices such as a phone, a user desktopcomputer, or a back drive in user's home, image service provider such asShutterfly™, third party services such as Facebook™, Instagram™,Pinterest™, and cloud storage services such as Amazon™, Google™,Dropbox™ Carbonite™, etc.

Secondly, the automatic identification and selection of images can bebased on the person(s) and the subject defined in the specificationparameters (i.e. matched specification terms). For example, if the imageproduct to be made is a family photobook or to be given to Grandma, onlyphotos of family members (potentially with emphases on grand children)will be first selected. The person(s) in the photos can be identifiedusing tags or keywords associated with the photos or faces recognized inthe photos. In another example, if a user command recites “a photobookfor Hawaii vacation”, then the photos taken in Hawaii will be firstselected for evaluation. The photo capture location or vacation can bedetermined using geo location tag, keywords, and album names inassociation with the photos.

Next, the images identified in the (designated) image sources are rankedbased on predetermined criteria such as image quality, relevance,colors, orientation, format, and similarity among images, etc. Forexample, a photo that is fuzzy or of low resolution has lower imagequality. The ranking can produce ranking values for the images. Detailsfor ranking images for image products and services are also disclosed incommonly assigned U.S. Pat. No. 7,467,222, the disclosure of which isincorporated herein by reference.

The images are then selected based on the ranking. Higher ranked imagesare selected. Lower quality and less relevant images are discarded.Similar or redundant photos can be trimmed. Similar to what is describedpreviously in relation to FIG. 2 (step 220) and FIG. 10 (step 1020), thenumber of images selected is dependent on the type of image product tobe created. In other words, a subset of the identified images areselected comprising an image number within a predetermined rangecompatible with the image product type For example, a 12 month bookcalendar typically includes a cover photo, and 1-5 photos per month sothe total number of images can be limited to less than 60, such as 30. Aphotobook on the hand should at least include 20 images. A Christmascard typically includes less than six photos, and most commonly, onlyone photo.

With product type, format, style and image selected, a design for theimage product is automatically created using the selected images and thestyle or format for the image product type (step 1155). The layout ofthe images in the image product can be determined capture times, geolocations, image properties (color, density, sharpness, tone scale,image quality, etc.). Examples of automatic image layout in imageproducts are described above in association with FIGS. 7-9.

The design of the image product can be presented to a user for preview(step 1160). The presentation can be made at a web user interface suchas in a web browser, in a mobile application, or in a design file, etc.The user can be allowed to edit and enter additional information such asbook title and captions within the pages. The user can decide to orderan image product based the image product design, or share the imageproduct design.

The recipient for which the image product is created can be defined bythe specification parameters obtained in the step of matching (step1130). For example, after being manufactured, a photobook made forgrandma is shipped to the grandma (the role of grandma is defined in thecontext of the user).

In some embodiments, as mentioned above in step 1115, the automatedcreation of personalized products can be initiated or triggered by therecognition of certain voice messages. These voice messages can includeexplicit user commands for the interest or the need for personalizedproducts. In some cases, users' intentions or potential needs forpersonalized products can be inferred from one or more voice messages.

Referring to FIG. 12, a library of specification terms for personalizedproducts is stored in a computer system (step 1210). The computer systemcan include a network based computer system, a user device, acombination thereof, or in other types of configuration. Thespecification terms cover all necessary information for creating apersonalized product. For example, the specification terms include thenames of personalized products or image services, or the types ofpersonalized products and services, such as photobook, greeting card,calendar, photo mugs, photo T-shirts, photo aprons, single photo pages,photo collage pages, a photo stationery, photo banners, photo mugs,photo mouse pads, photo key-chains, photo collectors, photo coasters, ora “photo story”, phone case, and various photo gifts and photo noveltyitems, etc. The library can store variations of the specification terms.For example, a photobook can also be referred to as a scrapbook or abook. A greeting card can have different variations such as a photogreeting card, a holiday card, a Christmas card, a birthday card, aValentine card, an invitation card, a thank you card, an announcementcard, a get-well card, etc. Other examples of personalized products andservices are available at websites operated by Shutterfly™, Tinyprints™,MyPublisher™, ThisLife™, etc.

The specification terms can also include contact names or equivalentassociated with a user or in a user's address book, which can be storedin the user's computer device or at a central location operated by aservice provider. The contact names can include family members, friends,and acquaintances. For example, the person grandma is defined in contextof the user. The contact names can also include faces recognized in theuser's images stored in the user device(s), at an image service providersuch as Shutterfly™, Inc., or at a third party service provider such asFacebook™, Instagram™, etc. In relation to personalized products, thecontact names can be the subjects in the photos to be incorporated in apersonalized product, or the recipients for whom the personalizedproduct is made.

The specification terms can include a time period within which theimages are to be selected. A photobook for the year 2013 usuallyincorporates images taken in 2013. Unless specified, a holiday cardrefers to the current holiday season or the holiday season coming up.The specification terms can include occasions for which a personalizedproduct is to be created: for example, Christmas, wedding, birthday,soccer game, anniversary, vacation, trip to the Caribbean's.

The types of personalized product can include photobooks, personalizedgreeting cards, photo stationeries, photo or image prints, photoposters, photo banners, photo playing cards, photo T-shirts, photo mugs,photo aprons, or magnets, mouse pads, phone cases, cases for tabletcomputers, key-chains, photo collectors, coasters, banners, or othertypes of photo gifts that include personalized content. Other examplesof personalized products include glassware, tiles, woodwork, jewelrytrays, paperweights, pillows, and mugs that include customizable graphicdesigns, text, or images.

The specification terms can include product styles such as modern,romantic, tropical, or styles developed by designer. The specificationterms can include color preferences such as warm, cool, blue, red,green, etc.

The specification terms can define the sources from which the imageswill be obtained. For example, the image sources can include userdevices such as a phone, a user desktop computer, or a back drive inuser's home, image service provider such as Shutterfly™, third partyservices such as Facebook™, Instagram™, Pinterest™, and cloud storageservices such as Amazon™, Google™, Dropbox™, Carbonite™, etc.

The specification terms can include one or more action words in one ormore voice messages from a user, which are related to a personalizedproduct. Examples of action words include “make”, “create”, “design”,“form”, “develop”, “produce”, “order”, “buy”, etc. These action wordsare often used at the beginning of a voice message, and can be used toclearly identify a voice message for creating a personalized product.

A voice message is received from a user (step 1215) by the computersystem. The computer system can include a computer (100 in FIG. 1), amobile phone, a tablet computer, a desktop computer connected to animage capture device, other computer or mobile communication devices, ora computer network with one or more terminals. The voice message can bereceived by a microphone (e.g. 106 in FIG. 1) that can be integrated ina computer device (such as a mobile phone, a tablet computer, laptopcomputer, etc.) or connected to a computer device as a peripheraldevice.

The voice message was first received as an analog audio signal by themicrophone, and converted to a digital signal in the I/O unit (117 inFigure). The voice message is typically in a natural language comprisingwords in a string of characters. The voice in the voice message in thedigital form is recognized, and converted into a text. Examples for avoice message include “Create a photobook for 2013”, or “Make a 2014photo calendar for grandma”, “Make a photobook for Peter's soccer team”,“Create mug with the last photo”, “Make holiday card using Christmasphotos”, “Create a photobook from Hawaii album”, “Make notepad forKatie”, “Make poster print from team photo”, “Create magnets using TangFamily photo”, “Make cards using Favorite photos (from phone photoalbum)”, or “Add selected photos to vacation photobook”.

A lexical analysis is conducted on the voice message (step 1220) by thecomputer system. The lexical analysis can include parsing or syntacticanalysis of the text in the voice message. The text string in the voicemessage is tokenized into a plurality of tokens (step 1225).

The tokens obtained from the voice message are then matched by thecomputer system to the specification terms in the library to determinespecification parameters for a personalized product (step 1230). Thestep of matching can include first converting the tokens to the standardforms to be consistent to the specification terms stored in the library,which can include removing articles “the” or “a” in the front of atoken, simplifying plural to a singular form, identifying synonyms ormorphological variations of a standard term and converting them tostandard forms. The step of matching then involves matching the tokensin their standard forms to the specification terms in the library.

The matched specification terms can determine specification parametersfor a personalized product. For example, tokens “photobook” and “2013”in the voice message “Create a photobook for 2013” define a photobookthat include photos taken in the year 2013. Some voice messages are incommand style and can include action words such as “make” or “create”,which can be used to identify a voice message for creating apersonalized product.

Optionally, the user can be asked by question(s) by the computer systemto define additional specification parameters for the personalizedproduct (step 1235). Some messages such as “Create a photobook” or “Makea birthday card” do not provide sufficient information to specify apersonalized product. The computer system can either automatically setparameters such as time period and the recipient or audience that is thephotobook is for, or can prompt question to the user to clarifyinformation such as the time period, the subject in the photos of thephotobook, whose birthday the card is for, from whom, etc.). Forexample, in response to a voice message “Create a photobook for 2013”,additional questions such as “For whom the photobook will be made?” or“Is the photobook for the family?” can be presented to the user. User'sanswers to these additional questions are analyzed similar to theoriginal voice message in steps 1215-1230 above.

A personalized product type is automatically determined by the computersystem based on the matched specification parameters that correspond toa product type such as “photobook” or “birthday card” in the library(step 1240).

Some voice messages may not include a command for making personalizedproducts, but their content can be used to infer the potential needs forpersonalized products. For example, if the voice message relates toplanning for a dinner out celebrating a spouse's birthday, the computersystem can infer the need for a birthday card or a personalized present,which in turn triggers the selections of a product type, content (e.g.personal photos), style or theme, and delivery time, etc. If voicemessages include discussions about a wedding, graduation, vacation, aball game, etc., personalized products related to these events can beautomatically selected (step 1240).

Additionally, a style or a format for the personalized product can beautomatically selected based on the specification parameters (step1245). For example, if the voice message or user answers to questionsinclude “8 by 8 inch photobook”, “wedding book”, “folded greeting card”,or “two piece iPhone 5s case”, these photo products can be respectivelyselected automatically.

Images are automatically identified and selected based on thespecification parameters (step 1250), which can include severalsub-steps.

First, images may be evaluated in a subset of or all available imagesources. If a voice message such as “Create a photobook for 2013” doesnot specify the sources of the images the personalized product is toincorporate, the images satisfying the criteria (e.g. taken in 2013)from all available image sources can be evaluated. Alternatively, thevoice message or answer may include image source specifications such as“Create a photobook for 2013 using pictures in my Shutterfly account”,or “Create a photobook for 2013 with photos from Shutterfly, Facebook,and Instagram”. Images will be retrieved and evaluated in the specifiedimage sources. As described above, image sources can include the imagesources can include user devices such as a phone, a user desktopcomputer, or a back drive in user's home, user accounts at image serviceprovider such as Shutterfly™, third party services such as Facebook™,Instagram™, Pinterest™, and cloud storage services such as Amazon′,Google™, Dropbox™, Carbonite™, etc.

Secondly, the automatic identification and selection of images can bebased on the person(s) and the subject defined in the specificationparameters (i.e. matched specification terms). For example, if thepersonalized product to be made is a family photobook or to be given toGrandma, only photos of family members (potentially with emphases ongrand children) will be first selected. The person(s) in the photos canbe identified using tags or keywords associated with the photos or facesrecognized in the photos. In another example, if a voice message recites“a photobook for Hawaii vacation”, then the photos taken in Hawaii willbe first selected for evaluation. The photo capture location or vacationcan be determined using geo location tag, keywords, and album names inassociation with the photos.

Next, the images identified in the (designated) image sources are rankedbased on predetermined criteria such as image quality, relevance,colors, orientation, format, and similarity among images, etc. Forexample, a photo that is fuzzy or of low resolution has lower imagequality. The ranking can produce ranking values for the images. Detailsfor ranking images for personalized products and services are alsodisclosed in commonly assigned U.S. Pat. No. 7,467,222, the disclosureof which is incorporated herein by reference.

The images are then selected based on the ranking. Higher ranked imagesare selected. Lower quality and less relevant images are discarded.Similar or redundant photos can be trimmed. Similar to what is describedpreviously in relation to FIG. 2 (step 220) and FIG. 10 (step 1020); thenumber of images selected is dependent on the type of personalizedproduct to be created. In other words, a subset of the identified imagesare selected comprising an image number within a predetermined rangecompatible with the personalized product type For example, a 12 monthbook calendar typically includes a cover photo, and 1-5 photos per monthso the total number of images can be limited to less than 60, such as30. A photobook on the hand should at least include 20 images. AChristmas card typically includes less than six photos, and mostcommonly, only one photo.

With product type, format, style and image selected, a design for thepersonalized product is automatically created using the selected imagesand the style or format for the personalized product type (step 1255).The layout of the images in the personalized product can be determinedcapture times, geo locations, image properties (color, density,sharpness, tone scale, image quality, etc.). Examples of automatic imagelayout in personalized products are described above in association withFIGS. 7-9.

The design of the personalized product is optionally presented to a userfor preview. The presentation can be made at a web user interface suchas in a web browser, in a mobile application, or in a design file, etc.The user can be allowed to edit and enter additional information such asbook title and captions within the pages. The user can decide to order apersonalized product based the personalized product design, or share thepersonalized product design. The displaying step for the design of thepersonalized product can include the creation and the rendering of athree-dimensional model for the personalized product including all thepersonalized content such as the selected photo, personalized designelements, text or a logo. The three-dimensional model is thepersonalized product can be displayed in perspective views on thedevice.

The user can review the personalized product at a user interface andorder it to be made into a physical product. A physical product can bemanufactured at a manufacturing facility (e.g. 170, FIG. 1) based on thedesign of the personalized product (step 1260).

The recipient for which the personalized product is created can bedefined by the specification parameters obtained in the step of matching(step 1230). For example, after being manufactured, a photobook made forgrandma is shipped to the grandma (the role of grandma is defined in thecontext of the user).

The design of the personalized product of the product type incorporatingthe selected photo can optionally be electronically shared share toanother device. The second device can for example be operated by afriend or family friend of the first user. The share can be initiated bythe first user who likes the design after preview and decides to shareit with his or her family member of friend. The electronic share can beconducted over a wired or wireless computer network, a local network, anintranet, or the Internet. The design of the personalized productincorporating the selected photo can be automatically displayed on thesecond device.

An advantage of the above-disclosed methods is that the users do notneed to go through a lengthy process to create a personalized imageproduct as in the convention systems. Another advantage of theabove-disclosed methods is that personalized products are automaticallycreated by recognizing voice messages without user input via a touchscreen or a keyboard.

What is claimed is:
 1. A computer system for proactively creating animage product, comprising: a computer memory configured to store alibrary of specification terms for image products; an I/O unitconfigured to receive a voice message from a user; and a computerprocessor in communication with the I/O unit and the computer memory,wherein the computer processor is configured to convert the voicemessage into a text form, to conduct lexical analysis of the text,tokenize the text into a plurality of tokens by the computer system,match one or more of the tokens to the specification terms in thelibrary to determine specification parameters for an image product,automatically identify images based on the specification parameters, andto automatically create a design for the image product that incorporatesat least some of the images identified based on the specificationparameters.
 2. The computer system of claim 1, wherein the specificationterms in the library specify one or more image sources from which theimages are to be obtained, wherein one or more of the tokens are matchedto the specification terms in the library that specify the one or moreimage sources, wherein the computer processor is further configured toautomatically identify comprises identifying images at the one or moreimage sources.
 3. The computer system of claim 1, wherein thespecification terms in the library specify one or more persons, whereinone or more of the tokens are matched to the specification terms in thelibrary that specify one or more persons, wherein the computer processoris further configured to automatically identifying comprises identifyingimages that include the one or more persons in the images.
 4. Thecomputer system of claim 1, wherein the specification terms in thelibrary specify an occasion that the images are associated with, whereinone or more of the tokens are matched to the specification terms in thelibrary that specify an occasion, wherein the computer processor isfurther configured to automatically identify comprises identifyingimages that are associated with the occasion.
 5. The computer system ofclaim 1, wherein the specification terms in the library specify a timeperiod in which the images are taken, wherein one or more of the tokensare matched to one or more specification terms in the library thatspecify a time period in which the images are taken, wherein thecomputer processor is further configured to automatically identifycomprises identifying images that are taken in the time period.
 6. Thecomputer system of claim 1, wherein the specification terms in thelibrary specify one or more image product types, wherein one or more ofthe tokens are matched to at least one specification term in the librarythat specifies an image product type, wherein the design for the imageproduct is automatically created in the image product type.
 7. Thecomputer system of claim 1, wherein the computer processor is furtherconfigured to select a subset of the identified images, wherein a numberof the identified images in the subset is within a predetermined rangecompatible with the image product type.
 8. The computer system of claim1, wherein the specification terms in the library specify a style or aformat for an image product, wherein one or more of the tokens arematched to at least one specification term in the library that specifiesa style or a format for an image product, wherein the design for theimage product is automatically created in the style or the format. 9.The computer system of claim 1, wherein the computer processor isfurther configured to design to a manufacturing facility in which aphysical manifestation of the image product manufactured based on thedesign for the image product.
 10. The computer system of claim 9,wherein the specification terms in the library specify a recipient foran image product, wherein one or more of the tokens are matched to aspecification term in the library that specifies a recipient for animage product, wherein the physical manifestation of the image productis sent to the recipient.
 11. The computer system of claim 1, whereinthe computer processor is further configured to rank the identifiedimages using predetermined criteria to produce a ranking result, andselect a subset of the identified images based on the ranking result,wherein the design for the image product incorporates the subset of theidentified images.
 12. The computer system of claim 1, wherein thecomputer processor is further configured to present a question to theuser after the one or more of the tokens are matched to thespecification terms in the library, to receive an answer from the user,to tokenize the answer into a plurality of tokens, and to match one ormore of the tokens to the specification terms in the library todetermine additional specification parameters for an image product. 13.The computer system of claim 1, wherein the computer processor isfurther configured to assign at least a portion of the identified imagesinto a first page group and a second page group each comprising one ormore images, to place the first page group of one or more images in thefirst page of the image product, and to place the second page group ofone or more images in the second page of the image product.
 14. Thecomputer system of claim 1, wherein the computer processor is furtherconfigured to conduct parsing or syntactic analysis of the text whenlexical analysis is conducted on the text.
 15. The computer system ofclaim 1, wherein the computer processor is further configured to conductparsing the text when the text is tokenized.
 16. The computer system ofclaim 1, wherein the computer processor is further configured to matchthe one or more of the tokens to morphological variations of thespecification terms in the library to determine specification parametersfor an image product.